Demands on computer systems to perform greater numbers of complex and critical functions has led to the distribution of functions, or portions thereof, to specialized systems. These specialized systems can then be optimized for specific tasks at the expense of being loosely coupled, which increases the complexities involved with system integration.
For some specialized systems, the complexity of full system integration has proved infeasible and has been avoided by redundantly maintaining selected data in various databases as well as maintaining redundant suites of applications to process that data. In an engineering company, for example, significant differences exist between systems for accounting and engineering groups. The accounting systems maintain data related to budget, man-hours, pay scales, and taxes, while the engineering systems maintain data related to engineering analyses, materials, designs, man-hours and budget. Although access to data regarding man-hours, budgets, and budgetary projections for the engineering groups by the accounting system is necessary, the costs involved with integrating these systems may be much more than costs involved with maintenance of redundant databases. In particular, raw data utilized to generate the projections for the engineering group may be useful to the accounting systems so complex programming and intimate knowledge of the engineering systems may be necessary to retrieve the raw data, which also adds a significant processing burden to the engineering systems. On the other hand, maintenance of redundant data increases the difficulty associated with ensuring that the data is correct and updated in a timely manner. Maintenance of redundant data also increases the difficulty associated with ensuring that processes such as budget projections, which are designed to manipulate the data, do so accurately.
One solution to combat the complexities involved with integration of loosely coupled, specialized systems is to incorporate a common interface such as a middleware application on each of the specialized systems. The middleware applications, such as IBM's message queuing Websphere™ applications, facilitate messaging systems that allow one system to gather data from another system.
The specialized systems can send messages to store data in queues of the middleware, and processing systems that are independent of, e.g., the accounting and engineering systems, can process and format the data. The message queuing applications can then store the data in, e.g., a data warehouse so the data can be accessed and utilized by any system having access privileges. Moreover, data that is reformatted and processed for the accounting systems may be stored in a database local to the accounting systems and data that is reformatted and processed for the engineering systems can be stored in a database local to the engineering systems, whether or not the accounting systems and/or engineering systems are in the same city, state, or even country.
As the messaging systems between the multiple, specialized systems become more prevalent and systems become more dependent upon the communications, system failures become a more critical problem, causing other, linked systems to hang or fail. In particular, an agent searching through messages in a message queue for data to, e.g., move to a data warehouse, may lose their pointer to the message queue, which may, for instance, contain hundreds of thousands of entries. Thus, after the failure, the agent must start from the beginning of the entries, searching through hundreds of thousands of entries and reformatting and processing thousands of entries again. Even worse, if the cause of the failure is a faulty entry at, e.g., record number 300,001, the failure may occur again once the agent reaches record 300,001.
One solution is to integrate routines in every agent to remember and store the current record that is being processed prior to processing the record. However, agents can be unique applications for each particular process and system. Thus, a large number of lines of code may be necessary to implement such protection for each and every agent associated with each and every specialized system.
Therefore, there is a need for tracking the record numbers of queues being accessed by an agent without significantly impacting code requirements for agents.